A CMOS image sensor with pixel level analog-to-digital conversion is described in U.S. Pat. No. 5,461,425 of B. Fowler et al. (the '425 patent). Such an image sensor, referred to as a digital pixel sensor (DPS), provides a digital output signal at each pixel element representing the light intensity detected by that pixel element. The combination of a photodetector and an analog-to-digital (A/D) converter in an area image sensor helps enhance detection accuracy and reduce power consumption, and improves overall system performance.
In the DPS array of the '425 patent, the analog-to-digital conversion (ADC) is based on first order sigma delta modulation. While this ADC approach requires fairly simple and robust circuits, it has the disadvantages of producing too much data and suffering from poor low light performance. U.S. Pat. No. 5,801,657 of Fowler et al., and U.S. patent application Ser. No. 09/274,202 provide alternative ADC mechanisms that can significantly improve the overall system performance while minimizing the size of the A/D converters. The aforementioned patents and patent application are incorporated herein by reference in their entireties.
An integrated DPS sensor may include an on-chip memory for storing at least one frame of pixel data. The incorporation of an on-chip memory in a DPS sensor alleviates the data transmission bottleneck problem associated with the use of an off-chip memory for storage of the pixel data. In particular, the integration of a memory with a DPS sensor makes feasible the use of multiple sampling for improving the quality of the captured images. Multiple sampling is a technique capable of achieving a wide dynamic range without many of the disadvantages associated with other dynamic range enhancement techniques, such as degradation in signal-to-noise ratio and increased implementation complexity. The aforementioned patent and patent applications are incorporated herein by reference in their entireties.
FIG. 1 shows a functional block diagram of an image sensor 300. The operation of image sensor 300 using multiple sampling is described as follows. Image sensor 300 includes a DPS sensor array 302 which has an N by M array of pixel elements. Sensor array 302 is similar to the digital pixel sensor described in the '425 patent and incorporates pixel level analog-to-digital conversion. A sense amplifier and latch circuit 304 is coupled to sensor array 302 to facilitate the readout of digital signals from sensor array 302. The digital signals (also referred to as digital pixel data) are stored in digital pixel data memory 310. To support multiple sampling, image sensor 300 also includes a threshold memory 306 and a time index memory 308 coupled to sensor array 302. Threshold memory 306 stores information of each pixel indicating whether the light intensity value measured by each pixel in sensor array 302 has passed a predetermined threshold level. In this example, the information is stored as a one-bit threshold indicator bit. The exposure time indicating when the light intensity measured by each pixel has passed the threshold level is stored in time index memory 308. In this example, the time index value is a two-bit value identifying each time exposure. As a result of this memory configuration, each pixel element in sensor array 302 can be individually time-stamped by threshold memory 306 and time index memory 308 and stored in digital pixel data memory 310.
With the memory configuration outlined above and illustrated in FIG. 1, image sensor 300 can implement multiple sampling to improve the quality of an image. In multiple sampling, each pixel element is exposed to an image at two or more different exposure times in order to compensate for bright and dark portions of the image. Additionally, the information regarding the exposure time associated with each pixel and the integrated intensity for that pixel is stored in time index memory 308 and digital memory 310 for use in computing the simulated pixel intensity when needed.
Sensor array 302 is an N by M array of pixels where each pixel outputs a digitized pixel voltage signal having k bits. Thus, the size of threshold memory 306 is N by M bits and the size of time index memory 308 is N by M by m bits where m is the number of bits representing the time index values. For example, when the resolution of sensor array 302 is 1024 by 1024 pixels, each pixel outputting 10 bits each (i.e., N=M=1024 and k=10), the size of threshold memory 306 is 1 megabits, the size of time index memory 308 with a 2-bit time index value is 2 megabits, and digital pixel data memory 310 is at least 10 megabits (or 1024×1024×10 bits) for storing one frame of image data.
To implement multiple sampling in an image sensor, memory space must be provided to store image information such as the threshold indicator bit and the time index value. When image sensor 300 in the example above is implemented in an integrated circuit, the size of the on-chip memory must be at least 13 megabits. If the resolution of the sensor array (i.e. the number of pixel elements) increases, the size of the on-chip memory will increase correspondingly. Integrating a large on-chip memory in an image sensor not only increases manufacturing cost but also adversely impacts yield. Therefore, it is desirable to minimize the size of the on-chip memory while supporting multiple sampling operations in a digital image sensor.